Min-max classifiers: Learnability, design and application

This paper introduces the class of min max classifiers. These are binary-valued functions that can be used as pattern classifiers of both real-valued and binary-valued feature vectors. They are also lattice-theoretic generalization of Boolean functions and are also related to feed-forward neural networks and morphological signal operators. We studied supervised learning of these classifiers under the Probably Approximately Correct (PAC) model proposed by Valiant. Several subclasses of thresholded min-max functions are shown to be learnable, generalizing the learnability results for the corresponding classes of Boolean functions. We also propose a LMS algorithm for the practical training of these pattern classifiers. Experimental results using the LMS algorithm for handwritten character recognition are promising. For example, in our experiments the min-max classifiers were able to achieve error rates that are comparable or better than those generated using neural networks. The major advantage of min max classifiers compared to neural networks is their simplicity and the faster convergence of their training algorithm. Pattern classification Character recognition Machine learning Mathematical morphology Image processing l. INTRODUCTION There is much interest in the field of pattern recognition on trainable pattern classifiers, as seen, for example, in the growth in the area of neural networks. Parallel to this development, in the field of machine learning there have been many theoretical advances on distribution-free learning of Boolean functions. This learning framework is known as the probably approximately correct (PAC) model, pioneered by Valiant (t) and further developed by him and other researchers. There is already a wealth of literature about the PAC learning model; examples include Valiant, (~'2) Blumer et al., ~3'4l Haussler, (s) Kearns et al. (6) Kearns, 17) Rivest (s) and Schapire. w) Most of the results in PAC learning deal with Boolean functions. If such functions are used as (Boolean) pattern classifiers, then the input features must be binary-valued. Although this may be sufficient for classifying high-level predicate-like features, most of the pattern recognition applications, such as computer speech and object recognition, involve real-valued feature vectors. In this paper, we present the class of rain-max classi-fiersl] and study methods of their automatic design. These classifiers can accept as inputs both real-valued and binary-valued feature vectors. Each input variable to these functions is in the range [0, 1], in contrast to {0, 1} for the Boolean classifiers. Moreover, these rain-max classifiers are natural generalizations of the Boolean functions, because they are based on MIN/MAX operations which …